Method for reducing crosstalk on an autostereoscopic display

ABSTRACT

The invention relates to a method for reducing crosstalk on an autostereoscopic display, wherein the display comprises an array of pixels lined with a view altering layer, such as a lenticular lens stack or parallax barrier, which display further comprises an eye tracking system for determining the position of the eyes of a viewer relative to the display, which method comprises the steps of: -defining a common nonlinear physical model for a view altering layer portion corresponding to a pixel or group of pixels, which nonlinear physical model has at least one variable for the position of the respective pixel or group of pixels relative to the display, a variable for the viewing position of the eyes of a viewer relative to the display and parameters related to the variables; -calibrating the autostereoscopic display by repeating for all pixels or group of pixels of the display, the steps of: +obtaining calibration data by observing the visibility of a pixel or group of pixels from at least two viewing positions; +fitting the calibration data on the nonlinear physical model for the respective view altering layer portion to obtain the parameters related to the variables; and+storing the parameters for the respective view altering layer portion; -controlling the pixels of the autostereoscopic display to display 3D images, wherein the controlling comprises at least the steps of: +determining the viewing position of the eyes of a viewer using the eye tracking system;+rendering 3D images from image data taking into account the position of the pixels or group of pixels relative to the viewing position, while correcting the 3D images per pixel or group of pixels using the common nonlinear physical model and the stored parameters for the view altering layer portion corresponding to the pixel of group of pixels.

The invention relates to a method for reducing crosstalk on anautostereoscopic display, wherein the display comprises an array ofpixels lined with a view altering layer, such as a lenticular lens stackor parallax barrier, which display further comprises an eye trackingsystem for determining the position of the eyes of a viewer relative tothe display.

With an autostereoscopic display it is possible to provide theexperience of three-dimensional images to a viewer without the viewerhaving to wear special glasses. The view altering layer, which istypically a lenticular lens stack or parallax barrier, provided on thedisplay allows for pixels on the display only to be seen by either theleft eye or right eye of a viewer. This provides the possibility to usethe display and create distinct images for the left eye and the righteye, such that the viewer will experience a three-dimensional image.

The created distinct images displayed on the autostereoscopic displayare based on a single viewing position of the viewer. As soon as theviewer shifts relative to the display, the images fall out of sync andthe perception of an three-dimensional image is lost. To this end it isknown to use an eye tracking system, which determines the position ofthe eyes, such that the generated image for the display can be adjustedto the position of the viewer.

Another aspect providing a disturbance in the perceivedthree-dimensional image is crosstalk. Due to the shape of the lenticularlens stack or parallax barrier, some light from a pixel will leak fromthe image of one eye to the other eye, such that in the image for theleft eye a ghost image of the right eye is present and vice versa. It isknown to compensate the image for the ghost image, such that crosstalkis reduced.

However, such crosstalk compensation is only effective if theautostereoscopic display is fully uniform and without any flaws. Due tomanufacturing tolerances, this will however not be the case, such thatthere is typically at least some crosstalk in the three-dimensionalimage displayed on the autostereoscopic display.

The manufacturing tolerances could result from a non-uniform pitch overthe lenticular lens stack, a non-uniform spacing between the lens stackand the pixels of the display, a non-uniform pitch between the pixels ofthe display or even between the sub-pixels, such that crosstalk couldeven differ between the three primary colors. Similar tolerances alsoapply to other principals for autostereoscopic displays, such asparallax barrier displays. Furthermore, reflections within the lensstack and lens aberrations could also lead to crosstalk.

US 2006 0268104 provides a method for further reducing crosstalk in aautostereoscopic display. In this publication it is suggested to apply acalibration step for the autostereoscopic display, such that the ghostimages can be reduced further. To obtain the calibration data thispublication refers to the method of U.S. Pat. No. 5,777,720. In U.S.Pat. No. 5,777,720 it is described that a test image is displayed andthat a viewer determines the optimal viewing position. This position isthen stored to be used in correcting the images depending on theposition of the viewer.

With this calibration method it is possible to provide a correctiondepending on the position of the eyes of the viewer. It is even possibleto provide separate correction values for groups of pixels or specificpixels of the display, but this will generate a substantial amount ofcalibration data.

This calibration data can be stored in tables, such that correctionvalues can be looked up during the rendering of the images. However, theamount of data, especially with a per pixel crosstalk compensation, issuch that a substantial amount of memory is needed and sufficientprocessing power to be able to provide the crosstalk correctionreal-time.

It is desired to be able to produce autostereoscopic displays to whichsimple image data can be supplied and wherein the display converts thisimage data in a three-dimensional image. As a result of the requiredprocessing power, the cost for the controller in the display would betoo high if the calibration method according to the prior art would beused. Especially, the amount of memory required to store the calibrationdata would increase the costs excessively.

Accordingly, it is an object of the invention to reduce or even removethe above mentioned disadvantages.

This object is achieved according to the invention with a method forreducing crosstalk on an autostereoscopic display, wherein the displaycomprises an array of pixels lined with a view altering layer, whichdisplay further comprises an eye tracking system for determining theposition of the eyes of a viewer relative to the display, which methodcomprises the steps of:

-   -   defining a common nonlinear physical model for a view altering        layer portion corresponding to a pixel or group of pixels, which        nonlinear physical model has at least one variable for the        position of the respective pixel or group of pixels relative to        the display, a variable for the viewing position of the eyes of        a viewer relative to the display and parameters related to the        variables;    -   calibrating the autostereoscopic display by repeating for all        pixels or group of pixels of the display, the steps of:        -   obtaining calibration data by observing the visibility of a            pixel or group of pixels from at least two viewing            positions;        -   fitting the calibration data on the nonlinear physical model            for the respective view altering layer portion to obtain the            parameters related to the variables; and        -   storing the parameters for the respective view altering            layer portion;    -   controlling the pixels of the autostereoscopic display to        display 3D images, wherein the controlling comprises at least        the steps of:        -   determining the viewing position of the eyes of a viewer            using the eye tracking system;        -   rendering 3D images from image data taking into account the            position of the pixels or group of pixels relative to the            viewing position, while correcting the 3D images per pixel            or group of pixels using the common nonlinear physical model            and the stored parameters for the view altering layer            portion corresponding to the pixel of group of pixels.

With the method according to the invention a common nonlinear physicalmodel for a view altering layer portion is defined, such thatcalibration data for a pixel or group of pixels can be fitted and alimited number of parameters only need to be stored. The storedparameters then can be used with the same common linear physical modelto reproduce the calibration data and to correct the rendered images.

As a result the amount of memory can be reduced drastically, reducingcosts and making a display with integrated crosstalk correctioneconomically viable.

Because crosstalk is at least dependent on the position of the viewerrelative to the display and on the position of the pixel or group ofpixels for which crosstalk is to be corrected, the nonlinear model hasat least as input these variables. The model further takes into accountthe physical shape and dimensions of the view altering layer and of thepixels.

The calibration data is obtained by observing the pixels or group ofpixels from at least two viewing positions, such that for a specificautostereoscopic display it is determined if a specific pixel or groupof pixels is viewable as expected, or that a correction is required dueto manufacturing tolerances. The observing can be done by a camera,which has a sufficient resolution to avoid moire effects, by zooming inon specific pixels of the display and/or defocusing the camera, as onlythe visibility of a pixel has to be observed. Also, changing the viewingposition can be automated by arranging the camera on a robot arm or thelike, which allows the camera to be positioned in the plurality ofviewing positions.

Preferably, an interpolation is used for a group of pixels to determinethe correction per pixel. Although a constant correction value for agroup of pixels could be used, having an interpolation would furtherincrease the accuracy of the crosstalk compensation.

In yet a further embodiment of the method according to the inventioneach pixel comprises at least two sub-pixels, preferably threesub-pixels. These three sub-pixels will typically produce the threeprimary colors, red, green and blue.

In a further preferred embodiment of the method according to theinvention the common nonlinear physical model furthermore comprises avariable and corresponding parameter corresponding to a specificsub-pixel and wherein during obtaining calibration data the visibilityof each sub-pixel within a pixel is observed.

As the sub-pixels are arranged adjacent to each other, the position ofeach sub-pixel will vary slightly relative to the display. As a resultof this slight difference in position, there is some difference in theghost image when split into the three primary colors. So, by correctingon a sub-pixel level, the crosstalk would be further reduced. In orderto obtain the correction values from the model, the model is providedwith a variable indicating the specific sub-pixel.

In yet another preferred embodiment of the method according to theinvention, the common nonlinear physical model furthermore comprises avariable and corresponding parameter corresponding to the six degrees offreedom movement of the eyes and wherein the eye tracking systemdetermines, during controlling the pixels of the display, said sixdegrees of freedom movement of the eyes of the viewer.

Both positions in three dimensions of each of the eyes, when viewing thedisplay, have influence on the observed crosstalk. If a viewer wouldlook from the corners of the eyes, then the effective distance betweenthe eyes would be less then when the viewer would look straight forward.Also, such changes have an effect on the perceived crosstalk and can becompensated for if the nonlinear physical model takes these changes intoaccount.

However, for a simplified method according to the invention, the centerbetween the two eyes of the viewer could be used.

These and other features of the invention will be elucidated inconjunction with the accompanying drawings.

FIG. 1 shows schematically the basic functioning of an autostereoscopicdisplay.

FIG. 2 shows schematically the autostereoscopic display of FIG. 1 witheye tracking and a camera for obtaining calibration data.

FIGS. 3A and 3B show a diagram of an embodiment of the method accordingto the invention.

FIG. 1 shows schematically an autostereoscopic display 1 having an arrayof pixels 2, 3 lined with a lenticular lens stack 4. The pixels 2, 3have typically three sub-pixels, like a red, green and blue sub-pixel.These sub-pixels are however not shown in the figures for clarity sake.

When a viewer V looks at the screen with the left eye 5 and right eye 6,the lenticular lenses of the lens stack 4 will direct the light of thepixels 2 towards the right eye 6, while the light of the pixels 3 isdirect into the left eye 5. So, when the viewer V is positionedcorrectly in front of the display 1, the left eye 5 will only see pixels3, while the right eye 6 will only see pixels 2. This enables one tocreate with the pixels 2 an image different from the image created withthe pixels 3. As a result the viewer V will experience athree-dimensional image.

FIG. 2 shows the display 1 with an eye tracking system 7 mounted abovethe display 1, to establish the position of the eyes 5, 6 of the viewerV relative to the display 1.

In order to obtain calibration data, a camera 8 is positioned in frontof the display 1 and observes a group of pixels 9. The observation datais processed according to the method of the invention, which will beelucidated in conjunction with FIGS. 3A and 3B.

After observing the group pixels 9, a next group of pixels is observedand processed, such that the full surface of the display 1 has beenobserved.

Then the camera 8 is moved to a next position, which can be in anythree-dimensional direction, and the display 1 is scanned and observedagain.

FIGS. 3A and 3B show a diagram 20 of the method according to theinvention, which diagram starts in FIG. 3A and continues in FIG. 3B.

In step 21 a common nonlinear physical model M for the lenticular lensstack portion 4 corresponding to a pixel 2,3 or group of pixels 9 isdefined. This model M has at least a variable for the position of apixel P_(p) and a variable for the position P of the eyes of the viewerV relative to the display 1. The model can be dependent on morevariables, such a sub-pixel position and the six degrees of freedommovement of the eyes, but for clarity a simple model M is shown in thefigures.

In the next step 22 the pixel 2, 3 or group of pixels 9 is observed, asexplained in FIG. 2, to obtain calibration data for a specific pixelfrom a specific viewing position.

After all calibration data is obtained for all pixels 2, 3 or group ofpixels 9 from a number of viewing positions, the calibration data for aspecific pixel 2, 3 or group of pixels 9 is fitted in step 23 on themodel M to obtain a number of parameters, estimating the calibrationdata for said specific pixel 2, 3 or group of pixels 9 dependent on theviewing position P. These parameters are stored in step 24 into a memory25.

Continuing in FIG. 3B, the stored parameters are now used in aproduction situation. According to the invention, the position of theviewer V relative to the display 1 is determined by the eye trackingsystem 7 in step 26.

This viewing position is then used in combination with the model M andthe parameters stored in the memory 25 to render 3D images from imagedata D in step 27. The rendered 3D images are used to control the pixels2, 3 of the display 1 in step 28.

Then the cycle of steps 26, 27, 28 is repeated with new image data D torender and display a new 3D image.

1. Method for reducing crosstalk on an autostereoscopic display, whereinthe display comprises an array of pixels lined with view altering layer,such as a lenticular lens stack or parallax barrier, which displayfurther comprises an eye tracking system for determining the position ofthe eyes of a viewer relative to the display, which method comprises thesteps of: defining a common nonlinear physical model for a view alteringlayer portion corresponding to a pixel or group of pixels, whichnonlinear physical model has at least one variable for the position ofthe respective pixel or group of pixels relative to the display, avariable for the viewing position of the eyes of a viewer relative tothe display and parameters related to the variables; calibrating theautostereoscopic display by repeating for all pixels or group of pixelsof the display, the steps of: obtaining calibration data by observingthe visibility of a pixel or group of pixels from at least two viewingpositions; fitting the calibration data on the nonlinear physical modelhe respective view altering layer portion to obtain the parametersrelated to the variables; and storing the parameters for the respectiveview altering layer portion; controlling the pixels of theautostereoscopic display to display 3D images, wherein the controllingcomprises at least the steps of: determining the viewing position of theeyes of a viewer sing the eye tracking system; rendering 3D images fromimage data taking into account the position of the pixels or group ofpixels relative to the viewing position, while correcting the 3D imagesper pixel or group of pixels using the common nonlinear physical modeland the stored parameters for the view altering layer portioncorresponding to the pixel of group of pixels.
 2. Method according toclaim 1, wherein for a group of pixels an interpolation is used todetermine the correction per pixel.
 3. Method according to claim 1,wherein each pixel comprises at least two sub-pixels, preferably threesub pixels.
 4. Method according to claim 3, wherein the common nonlinearphysical model furthermore comprises a variable and correspondingparameter corresponding to a specific sub-pixel and wherein duringobtaining calibration data the visibility of each sub-pixel within apixel is observed.
 5. Method according to claim 1, wherein the commonnonlinear physical model furthermore comprises a variable andcorresponding parameter corresponding to the six degrees of freedommovement of the eyes and wherein the eye tracking system determines,during controlling the pixels of the display, said six degrees offreedom movement of the eyes of the viewer.